Jeff Elman
نویسندگان
چکیده
The broad context of this study is the investigation of the nature of computation in recurrent networks (RNs). The current study has two parts. The first is to show that a RN can solve a problem that we take to be of interest (a counting task), and the second is to use the solution as a platform for developing a more general understanding of RNs as computational mechanisms. We begin by presenting the empirical results of training RNs on the counting task. The task ( ) is the simplest possible grammar that requires a PDA or counter. A RN was trained to predict the deterministic elements in sequences of the form * where n=1 to 12. After training, it generalized to n=18. Contrary to our expectations, on analyzing the hidden unit dynamics, we find no evidence of units acting like counters. Instead, we find an oscillator. We then explore the possible range of behaviors of oscillators using iterated maps and in the second part of the paper we describe the use of iterated maps for understanding RN mechanisms in terms of “activation landscapes”. This analysis leads to used an understanding of the behavior of network generated in the simulation study.
منابع مشابه
Regularities in the Acquisition of the English Past Tense
Finding Structure in Time by Jeff Elman, Department of Linguistics, UC San Diego Paper: Rules and Regularities in the Acquisition of the English Past Tense by Virginia Marchman, Department of Psychology, UC Berkeley and UC San Diego
متن کاملDynamic Replication in Wide Area Environments Using Message Logging
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
متن کاملGeneralization, simple recurrent networks, and the emergence of structure
The problem is that most human behaviors seem to lie somewhere between these two possibilities. Neither lists nor rules capture the richness of cognitive behaviors, which are populated by what Plaut, McClelland, Seidenberg, and Patterson have called “quasi-regular” domains. There are underlying generalities to most of what we do (or think), but the generalities are typically partial and tempere...
متن کاملWatching the Transients : Viewing a Simple Recurrent Network as a Limited Counter
Researchers in analog computation theory have shown that a recurrent neural network (RNN) can be built to simulate a Turing machine (Pollack, 1987b; Siegelmann & Sontag, 1995). Recently, we showed that it is possible to train RNNs which imple ment some aspects of analog computation theory-namely a network can develop trajectories that count symbols (Wiles & Elman, 1995). But what are the implic...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1998